Video coding system with low delay and method of operation thereof

ABSTRACT

A method of operation of a video coding system includes: receiving a video bitstream as a serial bitstream; extracting a video syntax from the video bitstream; extracting a low delay flag, a network abstraction layer (NAL) hypothetical reference decode (HRD) parameters present flag, and a video coding layer (VCL) HRD parameters present flag from the video syntax extracting a HRD syntax from the video bitstream based on the low delay flag, the NAL HRD parameters present flag, and the VCL HRD parameters present flag; extracting a temporal layer from the video bitstream based on the video syntax having the HRD syntax; and forming a video stream based on the temporal layer for displaying on a device.

CROSS-REFERENCE TO RELATED APPLICATION

The present application contains subject matter related to aconcurrently filed U.S. Patent Application by Munsi Haque, Kazushi Sato,Ali Tabatabai, and Teruhiko Suzuki entitled “VIDEO CODING SYSTEM WITHTEMPORAL LAYERS AND METHOD OF OPERATION THEREOF”. The relatedapplication is assigned to Sony Corporation and is identified by docketnumber 1014-061. The subject matter thereof is incorporated herein byreference in its entirety.

The present application contains subject matter related to aconcurrently filed U.S. Patent Application by Munsi Haque, Kazushi Sato,Ali Tabatabai, and Teruhiko Suzuki entitled “VIDEO CODING SYSTEM WITHTEMPORAL SCALABILITY AND METHOD OF OPERATION THEREOF”. The relatedapplication is assigned to Sony Corporation and is identified by docketnumber 1014-062. The subject matter thereof is incorporated herein byreference in its entirety.

The present application claims the benefit of U.S. Provisional PatentApplication Ser. No. 61/667,294 filed Jul. 2, 2012 and U.S. ProvisionalPatent Application Ser. No. 61/677,349 filed Jul. 30, 2012 and thesubject matter thereof is incorporated herein by reference in itsentirety.

TECHNICAL FIELD

The present invention relates generally to video systems, and moreparticularly to a system for video coding with low delay.

BACKGROUND ART

The deployment of high quality video to smart phones, high definitiontelevisions, automotive information systems, and other video deviceswith screens has grown tremendously in recent years. The wide variety ofinformation devices supporting video content requires multiple types ofvideo content to be provided to devices with different size, quality,and connectivity capabilities.

Video has evolved from two dimensional single view video to multiviewvideo with high-resolution three dimensional imagery. In order to makethe transfer of video more efficient, different video coding andcompression schemes have tried to get the best picture from the leastamount of data. The Moving Pictures Experts Group (MPEG) developedstandards to allow good video quality based on a standardized datasequence and algorithm. The H.264 (MPEG4 Part 10)/Advanced Video Codingdesign was an improvement in coding efficiency typically by a factor oftwo over the prior MPEG-2 format. The quality of the video is dependentupon the manipulation and compression of the data in the video. Thevideo can be modified to accommodate the varying bandwidths used to sendthe video to the display devices with different resolutions and featuresets. However, distributing larger, higher quality video, or morecomplex video functionality requires additional bandwidth and improvedvideo compression.

Thus, a need still remains for a video coding system that can delivergood picture quality and features across a wide range of device withdifferent sizes, resolutions, and connectivity. In view of theincreasing demand for providing video on the growing spectrum ofintelligent devices, it is increasingly critical that answers be foundto these problems. In view of the ever-increasing commercial competitivepressures, along with growing consumer expectations and the diminishingopportunities for meaningful product differentiation in the marketplace,it is critical that answers be found for these problems. Additionally,the need to save costs, improve efficiencies and performance, and meetcompetitive pressures, adds an even greater urgency to the criticalnecessity for finding answers to these problems.

Solutions to these problems have long been sought but prior developmentshave not taught or suggested any solutions and, thus, solutions to theseproblems have long eluded those skilled in the art.

DISCLOSURE OF THE INVENTION

The present invention provides a method of operation of a video codingsystem including: receiving a video bitstream as a serial bitstream;extracting a video syntax from the video bitstream; extracting a lowdelay flag, a network abstraction layer (NAL) hypothetical referencedecode (HRD) parameters present flag, and a video coding layer (VCL) HRDparameters present flag from the video syntax extracting a HRD syntaxfrom the video bitstream based on the low delay flag, the NAL HRDparameters present flag, and the VCL HRD parameters present flag;extracting a temporal layer from the video bitstream based on the videosyntax having the HRD syntax; and forming a video stream based on thetemporal layer for displaying on a device.

The present invention provides a video coding system, including: areceive module for receiving a video bitstream as a serial bitstream; aget syntax module, coupled to the receive module, for extracting a videosyntax from the video bitstream, extracting a low delay flag and anetwork abstraction layer (NAL) hypothetical reference decode (HRD)parameters present flag and a video coding layer (VCL) HRD parameterspresent flag from the video syntax, and extracting a HRD syntax from thevideo bitstream based on the low delay flag, the NAL HRD parameterspresent flag, and the VCL HRD parameters present flag; a decode module,coupled to the get syntax module, for extracting a temporal layer fromthe video bitstream based on the video syntax having the HRD syntax; anda display module, coupled to the decode module, for forming a videostream based on the temporal layer for displaying on a device.

Certain embodiments of the invention have other aspects in addition toor in place of those mentioned above. The aspects will become apparentto those skilled in the art from a reading of the following detaileddescription when taken with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a video coding system in an embodiment ofthe present invention.

FIG. 2 is an example of the video bitstream.

FIG. 3 is an example of a HRD syntax.

FIG. 4 is an example of a High Efficiency Video Coding (HEVC) VideoUsability Information (VUI) syntax.

FIG. 5 is an example of a HEVC VUI extension syntax.

FIG. 6 is an example of a HRD base syntax.

FIG. 7 is an example of a HRD sub-layer syntax.

FIG. 8 is an example of a HRD VUI syntax.

FIG. 9 is a functional block diagram of the video coding system.

FIG. 10 is a control flow of the video coding system.

FIG. 11 is a flow chart of a method of operation of the video codingsystem in a further embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

The following embodiments are described in sufficient detail to enablethose skilled in the art to make and use the invention. It is to beunderstood that other embodiments would be evident based on the presentdisclosure, and that process or mechanical changes may be made withoutdeparting from the scope of the present invention.

In the following description, numerous specific details are given toprovide a thorough understanding of the invention. However, it will beapparent that the invention may be practiced without these specificdetails. In order to avoid obscuring the present invention, somewell-known circuits, system configurations, and process steps are notdisclosed in detail.

Likewise, the drawings showing embodiments of the system aresemi-diagrammatic and not to scale and, particularly, some of thedimensions are for the clarity of presentation and are shown greatlyexaggerated in the drawing FIGs. Where multiple embodiments aredisclosed and described, having some features in common, for clarity andease of illustration, description, and comprehension thereof, similarand like features one to another will ordinarily be described with likereference numerals.

The term “syntax” means the set of elements describing a data structure.The term “module” referred to herein can include software, hardware, ora combination thereof in the present invention in accordance with thecontext used.

Referring now to FIG. 1, therein is shown a block diagram of a videocoding system 100 in an embodiment of the present invention. A videoencoder 102 can receive a video content 108 and send a video bitstream110 to a video decoder 104 for decoding and display on a displayinterface 120.

The video encoder 102 can receive and encode the video content 108. Thevideo encoder 102 is a unit for encoding the video content 108 into adifferent form. The video content 108 is defined as a digitalrepresentation of a scene of objects. For example, the video content 108can be the digital output of one or more digital video cameras.

Encoding is defined as computationally modifying the video content 108to a different form. For example, encoding can compress the videocontent 108 into the video bitstream 110 to reduce the amount of dataneeded to transmit the video bitstream 110.

In another example, the video content 108 can be encoded by beingcompressed, visually enhanced, separated into one or more views, changedin resolution, changed in aspect ratio, or a combination thereof. Inanother illustrative example, the video content 108 can be encodedaccording to the High-Efficiency Video Coding (HEVC)/H.265 standard.

The video encoder 102 can encode the video content 108 to form the videobitstream 110. The video bitstream 110 is defined a sequence of bitsrepresenting information associated with the video content 108. Forexample, the video bitstream 110 can be a bit sequence representing acompression of the video content 108.

In another example, the video bitstream 110 can be a serial bitstream122. The serial bitstream 122 is a series of bits representing the videocontent 108 where each bit is transmitted serially over time.

The video encoder 102 can receive the video content 108 for a scene in avariety of ways. For example, the video content 108 representing objectsin the real-world can be captured with a video camera, multiple cameras,generated with a computer, provided as a file, or a combination thereof.

The video content 108 can include a variety of video features. Forexample, the video content 108 can include single view video, multiviewvideo, stereoscopic video, or a combination thereof. In a furtherexample, the video content 108 can be multiview video of four or morecameras for supporting three-dimensional (3D) video viewing without 3Dglasses.

The video encoder 102 can encode the video content 108 using a videosyntax 114 to generate the video bitstream 110. The video syntax 114 isdefined as a set of information elements that describe a coding systemfor encoding and decoding the video content 108. The video bitstream 110is compliant with the video syntax 114, such as High-Efficiency VideoCoding/H.265, and can include a HEVC video bitstream, an Ultra HighDefinition video bitstream, or a combination thereof. The videobitstream 110 can include the video syntax 114.

The video bitstream 110 can include information representing the imageryof the video content 108 and the associated control information relatedto the encoding of the video content 108. For example, the videobitstream 110 can include an occurrence of the video syntax 114 and anoccurrence of the video content 108.

The video coding system 100 can include the video decoder 104 fordecoding the video bitstream 110. The video decoder 104 is defined as aunit for receiving the video bitstream 110 and modifying the videobitstream 110 to form a video stream 112.

The video decoder 104 can decode the video bitstream 110 to form thevideo stream 112 using the video syntax 114. Decoding is defined ascomputationally modifying the video bitstream 110 to form the videostream 112. For example, decoding can decompress the video bitstream 110to form the video stream 112 formatted for displaying on the display thedisplay interface 120.

The video stream 112 is defined as a computationally modified version ofthe video content 108. For example, the video stream 112 can include amodified occurrence of the video content 108 with different resolution.The video stream 112 can include cropped decoded pictures from the videocontent 108.

In a further example, the video stream 112 can have a different aspectratio, a different frame rate, different stereoscopic views, differentview order, or a combination thereof than the video content 108. Thevideo stream 112 can have different visual properties includingdifferent color parameters, color planes, contrast, hue, or acombination thereof.

The video coding system 100 can include a display processor 118. Thedisplay processor 118 can receive the video stream 112 from the videodecoder 104 for display on the display interface 120. The displayinterface 120 is a unit that can present a visual representation of thevideo stream 112.

For example, the display interface 120 can include a smart phonedisplay, a digital projector, a DVD player display, or a combinationthereof. Although the video coding system 100 shows the video decoder104, the display processor 118, and the display interface 120 asindividual units, it is understood that the video decoder 104 caninclude the display processor 118 and the display interface 120.

The video encoder 102 can send the video bitstream 110 to the videodecoder 104 over a communication path 106. The communication path 106can be a variety of networks suitable for data transfer.

In an illustrative example, the video coding system 100 can includecoded picture buffers (not shown). The coded picture buffers can act asfirst-in first-out buffers containing access units, where each accessunit can contain one frame of the video bitstream 110.

In another illustrative example, the video coding system 100 can includea hypothetical reference decoder (not shown). The hypothetical referencedecoder can be a decoder model used to constrain the variability of thevideo bitstream 110.

For example, the communication path 106 can include wirelesscommunication, wired communication, optical, ultrasonic, or thecombination thereof. Satellite communication, cellular communication,Bluetooth, Infrared Data Association standard (IrDA), wireless fidelity(WiFi), and worldwide interoperability for microwave access (WiMAX) areexamples of wireless communication that can be included in thecommunication path 106. Ethernet, digital subscriber line (DSL), fiberto the home (FTTH), and plain old telephone service (POTS) are examplesof wired communication that can be included in the communication path106.

The video coding system 100 can employ a variety of video coding syntaxstructures. For example, the video coding system 100 can encode anddecode video information using High Efficiency Video Coding/H.265. Thevideo coding syntaxes are described in the following documents that areincorporated by reference in their entirety:

B. Bross, W. Han, J Ohm, G. Sullivan, T. Wiegand, “High-Efficiency VideoCoding (HEVC) text specification draft 8”, JCTVC-J1003 d7, July 2012(Stockholm).

B. Bross, W. Han, J. Ohm, G. Sullivan, T. Wiegand, “High EfficiencyVideo Coding (HEVC) text specification draft 7” JCTVC-I1003 d4, May 2012(Geneva).

M. Haque, K. Sato, A. Tabatabai, T. Suzuki, “A simple ordering issue forVUI parameters syntax”, JCTVC-J0273, July 2012 (Stockholm).

M. Haque, K. Sato, A. Tabatabai, T. Suzuki, “HEVC VUI Parameters withExtension Hooks”, JCTVC-J0270, July 2012 (Stockholm).

M. Haque, A. Tabatabai, “Extension of HEVC VUI Syntax Structure”,JCTVC-10263, May 2012.

M. Haque, “AHG10: VUI and HRD syntax designs agreed by the BoG on VPSand NUH”, JCTVC-J0548r1, July 2012.

Referring now to FIG. 2, therein is shown an example of the videobitstream 110. The video bitstream 110 includes an encoded occurrence ofthe video content 108 of FIG. 1 and can be decoded using the videosyntax 114 to form the video stream 112 of FIG. 1 for display on thedisplay interface 120 of FIG. 1.

The video bitstream 110 can include a variety of video types asindicated by a syntax type 202. The syntax type 202 is defined as anindicator of the type of video coding used to encode and decode thevideo bitstream 110. For example, the video content 108 can include thesyntax type 202 for advanced video coding 204 (AVC), scalable videocoding 206 (SVC), multiview video coding 208 (MVC), multiview video plusdepth 210 (MVD), and stereoscopic video 212 (SSV).

The advanced video coding 204 and the scalable video coding 206 can beused to encode single view based video to form the video bitstream 110.The single view-based video can include the video content 108 generatefrom a single camera.

The multiview video coding 208, the multiview video plus depth 210, andthe stereoscopic video 212 can be used to encode the video content 108having two or more views. For example, multiview video can include thevideo content 108 from multiple cameras.

The video syntax 114 can include an entry count 216 for identifying thenumber of entries associated with each frame in the video content 108.The entry count 216 is the maximum number of entries represented in thevideo content 108.

The video syntax 114 can include an entry identifier 214. The entryidentifier 214 is a value for differentiating between multiple codedvideo sequences. The coded video sequences can include occurrences ofthe video content 108 having a different bit-rate, frame-rate,resolution, or scalable layers for a single view video, multiview video,or stereoscopic video.

The video syntax 114 can include an iteration identifier 218. Theiteration identifier 218 is a value to differentiate between individualiterations of the video content 108.

The video syntax 114 can include an iteration count 220. The iterationcount 220 is a value indicating the maximum number of iterations of thevideo content 108.

For scalable video coding, the term iteration count can be used toindicate the number of information entries tied to different scalablevideo layers in the case of scalable video coding. For multiview videocoding, the iteration count can be used to indicate the number ofoperation points tied to the number of views of the video content 108.

For example, in scalable video coding, the video content 108 can beencoded to include a base layer with additional enhancement layers toform multi-layer occurrences of the video bitstream 110. The base layercan have the lowest resolution, frame-rate, or quality.

The enhancement layers can include gradual refinements with additionalleft-over information used to increase the quality of the video. Thescalable video layer extension can include a new baseline standard ofHEVC that can be extended to cover scalable video coding.

The video syntax 114 can include an operation identifier 222. Theoperation identifier 222 is a value to differentiate between individualoperation points of the video content 108. The operation points areinformation entries present for multiview video coding, such as timinginformation, network abstraction layer (NAL) hypothetical referenceddecoder (HRD) parameters, video coding layer (VCL) HRD parameters, apic_struct_present flag element, or a combination thereof.

The video syntax 114 can include an operation count 224. The operationcount 224 is a value indicating the maximum number of operations of thevideo content 108.

The operation points are tied to generation of coded video sequencesfrom various views, such as views generated by different cameras, formultiview and 3D video. For multiview video coding, an operation pointis associated with a subset of the video bitstream 110 having a targetoutput view and the other views dependent on the target output view.

The other views are dependent on the target output view if they arederived using a sub-bitstream extraction process. More than oneoperation point may be associated with the same subset of the videobitstream 110. For example, decoding an operation point refers to thedecoding of the subset of the video bitstream corresponding to theoperation point and subsequent output of the target output views as aportion of the video stream 112 of FIG. 1 for display on the devicevideo encoder.

The video syntax 114 can include a view identifier 226. The viewidentifier 226 is a value to differentiate between individual views ofthe video content 108.

The video syntax 114 can include a view count 228. The view count 228 isa value indicating the maximum number of views of the video content 108.

For example, a single view can be a video generated by a single camera.Multiview video can be generated by multiple cameras situated atdifferent positions and distances from the objects being viewed in ascene.

The video content 108 can include a variety of video properties. Forexample, the video content 108 can be high resolution video, such asUltra High Definition video. The video content 108 can have a pixelresolution greater than or equal to 3840 pixels by 2160 pixels orhigher, including resolutions of 7680 by 4320, 8K by 2K, 4K by 2K, or acombination thereof. Although the video content 108 supports highresolution video, it is understood that the video content 108 can alsosupport lower resolutions, such as high definition (HD) video. The videosyntax 114 can support the resolution of the video content 108.

The video content 108 can support a variety of frame rates including 15frames per second (fps), 24 fps, 25 fps, 30 fps, 50 fps, 60 fps, and 120fps. Although individual frame rates are described, it is understoodthat the video content 108 can support fixed and variable frame rates ofzero frames per second and higher. The video syntax 114 can support theframe rate of the video content 108.

The video bitstream 110 can include one or more temporal layers 230. Thetemporal layers 230 are defined as portions of the video bitstream 110representing the video stream 112 at a specified frame rate. Each of thetemporal layers 230 can represent the video stream 112 at a differentframe rate expressed as frames per second (fps). The temporal layers 230can form a hierarchy with higher layers including the lower layers.

For example, a first occurrence 232 of the temporal layers 230 canrepresent a 15 fps occurrence of the video stream 112, a secondoccurrence 234 of the temporal layers 230 can represent a 30 fpsoccurrence of the video stream 112, and a third occurrence 236 of thetemporal layers 230 can represent a 60 fps occurrence of the videostream 112. The video bitstream 110 can have multiple occurrences of thetemporal layers 230 as indicated by a temporal layer count 238. In afurther example, the temporal layer count 238 can have a value of threefor the first occurrence 232, the second occurrence 234, and the thirdoccurrence 236.

The first occurrence 232 of the temporal layers 230 can represent a baselayer that encodes the video content 108 to form the video stream 112 at15 fps. The second occurrence 234 of the temporal layers 230 canrepresent the difference between the base layer, such as the firstoccurrence 232 of the temporal layers 230, and the video stream 112 ofthe video content 108 at 30 fps.

The second occurrence 234 can includes frames that represent thedifference between the frames of the base layer and the new framesrequired for displaying the video content 108 at 30 fps. The thirdoccurrence 236 of the temporal layers 230 can represent the differencebetween the second occurrence 234 of the temporal layers 230 and thevideo content at 60 fps.

In an illustrative example, the video decoder 102 of FIG. 1 for a smartphone can extract the second occurrence 234 of the temporal layers 230at 30 fps from the video bitstream 110, which can include theinformation from the first occurrence 232 and the second occurrence 234.The information in the video bitstream 110 from the third occurrence 236of the temporal layers 230 can be discarded to reduce the size of thevideo bitstream 110.

Referring now to FIG. 3, therein is shown an example of a HRD syntax302. The HRD syntax 302 describes the parameters associated with thehypothetical reference decoder.

The HRD syntax 302 includes elements as described in the HRD base syntaxtable of FIG. 4. The elements of the HRD syntax 302 are arranged in ahierarchical structure as described in the HRD base syntax table of FIG.4.

The HRD syntax 302 can be included in the video bitstream 110 of FIG. 1and delivered in a time serial manner with each element received in theorder it was sent. For example, the HRD syntax 302 can be received bythe video decoder 104 of FIG. 1 as a serial bitstream having theelements of the HRD syntax 302 delivered in the order described in theHRD syntax table of FIG. 4.

The HRD syntax 302 can include a HRD syntax header 304, such as thehrd_parameters element. The HRD syntax header 304 is a descriptor foridentifying the HRD syntax 302.

The HRD syntax 302 can include a coded picture buffer (CPB) count 308,such as a cpb_cnt_minus1 element. The CPB count 308 can indicate thenumber of alternative delivery schedules having restricted bit rates andCPB size values.

The HRD syntax 302 can include a bit rate scale 310, such as abit_rate_scale element. The bit rate scale 310 specifies the maximuminput bit rate of the CPB.

The HRD syntax 302 can include a CPB size scale 312, such as acpb_size_scale element. The CPB size scale 312 is for determining thesize of the coded picture buffer.

The HRD syntax 302 can include a loop structure to define a set ofparameters for each occurrence of the coded picture buffer. The loopstructure is dimensioned based on a schedule selection index 314, suchas a SchedSelldx element. The HRD syntax 302 loop structure can includea bit rate value 316, a CPB size value 318, and a constant bit rate(CBR) flag 320 for each occurrence of the coded picture buffer.

The HRD syntax 302 can include the bit rate value 316, such as abit_rate_value_minus1 element. The bit rate value 316 can be used tospecify the maximum input bit rate for each occurrence of the codedpicture buffer.

The HRD syntax 302 can include the CPB size value 318, such as acpb_size_value_minus1 element. The CPB size value 318 can be used todetermine the size of each occurrence of the coded picture buffer.

The HRD syntax 302 can include the CBR flag 320, such as a cbr flagelement. The CBR flag 320 indicates the operation mode for decoding thevideo bitstream 110 for each occurrence of the coded picture buffer. Ifthe CBR flag 320 has a value of 1, then the hypothetical stream deliveryschedule (HSS) operates in a constant bit rate mode. Otherwise, thevideo bitstream 110 operates in an intermittent bit rate mode.

The HRD syntax 302 can include an initial CPB removal delay length 322,such as an initial_cpb_removal_delay_length_minus1 element. The initialCPB removal delay length 322 indicates the bit length of the elementsinitial_cpb_removal_delay and initial_cpb_removal_delay_offset of thebuffering period supplemental enhancement information (SEI) message.

The HRD syntax 302 can include a CPB removal delay length 324, such as acpb_removal_delay_length_minus1 element. The CPB removal delay length324 can specify the bit length of the elements cpb_removal_delay in thepicture timing SEI message.

The HRD syntax 302 can include a decoded picture buffer (DPB) outputdelay length 326, such as a dpb_output_delay_length_minus1 element. TheDPB output delay length 326 indicates the size of the DPB.

The HRD syntax 302 can include a time offset length 328, such as atime_offset_length element. The time offset length 328 indicates thelength in bits of the time_offset_element.

Referring now to FIG. 4, therein is shown an example of a HighEfficiency Video Coding (HEVC) Video Usability Information (VUI) syntax402. The HEVC VUI syntax 402 includes information about the videobitstream 110 of FIG. 1 to permit additional application usabilityfeatures for the video content 108 of FIG. 1.

The HEVC VUI syntax 402 describes the elements in the HEVC VUI syntaxtable of FIG. 3. The elements of the HEVC VUI syntax 402 are arranged ina hierarchical structure as described in the HEVC VUI syntax table ofFIG. 3.

The HEVC VUI syntax 402 includes a HEVC VUI syntax header 404, such as avui_parameters element. The HEVC VUI syntax header 404 is a descriptorfor identifying the HEVC VUI syntax 402. The HEVC VUI syntax 402 is usedto encode and decode the video bitstream 110.

The HEVC VUI syntax 402 can include an aspect ratio flag 406, such asthe aspect_ratio_info_present_flag element. The aspect ratio flag 406can indicate that aspect ratio information is encoded in the videobitstream 110. The aspect ratio flag 406 can have a value 0 to indicatethat aspect ratio information is not in the video bitstream 110 and avalue of 1 to indicate that aspect ratio information is included in thevideo bitstream 110.

The HEVC VUI syntax 402 can include an aspect ratio indicator 408, suchas the aspect_ratio_idc element. The aspect ratio indicator 408 is avalue describing an aspect ratio of the video content 108 of FIG. 1. Forexample, the aspect ratio indicator 408, can include an index value foran enumerated list of predefined aspect ratios for the video content108. In a further example, the aspect ratio indicator 408 can include avalue indicating that the aspect ratio can be described by individualvalues for an aspect ratio width 410 and an aspect ratio height 412.

The HEVC VUI syntax 402 can include the aspect ratio width 410, such asthe sar_width element, The aspect ratio width 410 can describe the widthof the video content 108. The aspect ratio width 410 can describe thedimensions of the video content in ratios, pixels, lines, inches,centimeters, or a combination thereof.

The HEVC VUI syntax 402 can include the aspect ratio height 412, such asthe sar_height element. The aspect ratio height 412 can describe theheight of the video content 108. The aspect ratio height 412 candescribe the dimensions of the video content in ratios, pixels, lines,inches, centimeters, or a combination thereof.

The HEVC VUI syntax 402 can include an overscan present flag 414, suchas the overscan_info_present_flag. The overscan present flag 414 canindicate if overscan information is included in the video bitstream 110.The overscan present flag 414 can have a value of 1 to indicate thatoverscan information is present in the video bitstream or a value of 0to indicate that overscan information is not present in the videobitstream 110.

Overscan is defined as display processes in which some parts near theborders of the cropped decoded pictures of the video stream 112 of FIG.1 are not visible in the display area of the video stream 112. Underscanis defined as display processes in which the entire cropped decodedpictures of the video stream 112 are visible in the display area, but donot cover the entire display area.

The HEVC VUI syntax 402 can include an overscan appropriate flag 416,such as an overscan_appropriate_flag element. The overscan appropriateflag 416 can indicate that the video content 108 encoded in the videobitstream 110 can be displayed using overscan.

The overscan appropriate flag 416 can have a value of 1 to indicate thatthe cropped decoded pictures of the video stream 112 are suitable fordisplay using overscan. The overscan appropriate flag 416 can have avalue of zero to indicate that the cropped decoded pictures of the videostream 112 contain visually important information and should not bedisplayed using overscan.

The HEVC VUI syntax 402 can include a video signal present flag 418,such as the video_signal_type_present_flag element. The video signalpresent flag 418 can indicate that video signal type information isincluded in the video bitstream 110. The video signal present flag 418can have a value of 1 to indicate that additional video signal typeinformation is present in the video bitstream 110. The video signalpresent flag 418 can have a value of 0 to indicate that no video signaltype information is present in the video bitstream 110.

The HEVC VUI syntax 402 can include a video format 420, such as thevideo_format_element. The video format 420 can indicate the format ofthe video.

The HEVC VUI syntax 402 can include a video full range flag 422, such asthe video_full_range_flag element. The video full range flag 422 canindicate the black level and the range of the luma and chroma signalsfor the video content 108 encoded in the video bitstream 110.

The HEVC VUI syntax 402 can include an color description present flag424, such as the colour_description_present_flag element. The colordescription present flag 424 can indicate the presence of colordescription information in the video bitstream 110.

The color description present flag 424 can have a value of 0 to indicatethat no other color description information is included in the videobitstream 110. The color description present flag 424 can have a valueof 1 to indicate that a color primaries 426, a transfer characteristics428, and a matrix coefficient 430 are included in the video bitstream110.

The HEVC VUI syntax 402 can include the color primaries 426, such as thecolour_primaries element. The color primaries 426 can indicate the colorscheme used in the video content 108. For example, the color primaries426 can indicate the chromaticity coordinates of the source primaries.

The HEVC VUI syntax 402 can include the transfer characteristics 428,such as the transfer_characteristics element. The transfercharacteristics 428 can indicate the opto-electronic transfercharacteristics of the video content 108. For example, the transfercharacteristics 428 can be an enumerated value describing a predefinedset of display characteristics.

The HEVC VUI syntax 402 can include the matrix coefficient 430, such asthe matrix_coefficient element. The matrix coefficient 430 can indicatecoefficient used to derive luma and chroma signals from the red, green,and blue primaries indicated by the color primaries 426. The matrixcoefficient 430 can be used to computationally transform a set of red,blue, and green color coordinates to luma and chroma equivalents.

The HEVC VUI syntax 402 can include a chroma location informationpresent flag 432, such as the chroma_loc_info_present_flag element. Thechroma location information present flag 432 can have a value of 1 toindicate that a chroma top field sample 434 and a chroma bottom fieldsample 436 are present in the video bitstream 110.

The HEVC VUI syntax 402 can include the chroma top field sample 434,such as the chroma_sample_loc_type_top_field element. The chroma topfield sample 434 is an enumerated value to specify the location ofchroma samples for the top field in the video bitstream 110.

The HEVC VUI syntax 402 can include the chroma bottom field sample 436,such as the chroma_sample_loc_type_bottom_field element. The chromabottom field sample 436 is an enumerated value to specify the locationof chroma samples for the bottom field in the video bitstream 110.

The HEVC VUI syntax 402 can include a neutral chroma flag 438, such asthe neutral_chroma_indication_flag element. The neutral chroma flag 438can indicate whether the decoded chroma samples are equal to one. Forexample, if the neutral chroma flag 438 has a value of 1, then all ofthe decoded chroma samples are set to 1. If the neutral chroma flag 438has a value of 0, then the decoded chroma samples are not limited to 1.

The HEVC VUI syntax 402 can include a field sequence flag 440, such asthe field_seq_flag, can indicate whether coded video sequenceinformation includes video representing fields. The field sequence flag440 can have a value of 1 to indicate the coded video sequence of thevideo bitstream 110 includes field level pictures, and a value of 0 toindicate frame level pictures.

The HEVC VUI syntax 402 can include a timing information present flag442, such as the timing_info_present_flag element. The timinginformation present flag 442 can indicate whether timing information isincluded in the video bitstream 110. For example, the timing informationpresent flag 442 can have a value of 1 to indicate a tick units 444, atime scale 446, and a fixed picture rate flag 448 are present in thevideo bitstream 110.

The HEVC VUI syntax 402 can include the tick units 444, such as thenum_unit_in_tick element. The tick units 444 can indicate the number oftime units of a clock operating at the frequency of the time scale 446.For example, the tick units 444 can have corresponding to the minimuminterval of time that can be represented in the video bitstream 110.

The HEVC VUI syntax 402 can include the time scale 446, such as thetime_scale element. The time scale 446 is the number of time units thatpass in one second.

The HEVC VUI syntax 402 can include the fixed picture rate flag 448,such as the fixed_pic_rate_flag element. The fixed picture rate flag 448can indicate the whether the temporal distance between two consecutivepictures in the output order of the video stream 112 is constrained. Thefixed picture rate flag 448 has a value of 0 to indicate that noconstraint applies and a value of 1 to indicate that the temporaldistance is constrained.

The HEVC VUI syntax 402 can include a NAL HRD parameters present flag452, such as a nal_hrd_parameters_present_flag element. The NAL HRDparameters present flag 452 can indicate the presence of the NAL HRDparameters are included in the HRD syntax 302 of FIG. 3. The NAL HRDparameters present flag 452 can have a value of 1 to indicate that a HRDparameters structure 454 is present and a value of 0 to indicate the HRDparameters structure 454 is not present in the video bitstream 110.

The HEVC VUI syntax 402 can include the HRD parameters structure 454.The HRD parameters structure 454 is an occurrence of the HRD syntax 302of FIG. 3. The HRD parameters structure 454 is described in detail inthe HRD syntax section.

The HEVC VUI syntax 402 can include a VCL HRD parameters present flag456, such as a vcl_hrd_parameters_present_flag element, can indicate thepresence of the HRD information for VCL. The VCL HRD parameters presentflag 456 can have a value of 1 to indicate that the HRD parametersstructure 454 is present and a value of 0 to indicate the HRD parametersstructure 454 is not present in the video bitstream 110.

If the NAL HRD parameters present flag 452 or the VCL HRD parameterspresent flag 456 have a value of 1, then the HEVC VUI syntax 402 caninclude a low delay flag 460 and a sub-picture CPB parameters presentflag 462 in the video bitstream 110. If the sub-picture CPB parameterspresent flag 462 has a value of 1, then the HEVC VUI syntax 402 caninclude a subunit ticks 464 in the video bitstream 110.

The HEVC VUI syntax 402 can include the low delay flag 460, such as alow_delay_hrd_flag element. The low delay flag 460 can indicates the HRDoperational mode.

The HEVC VUI syntax 402 can include the sub-picture CPB parameterspresent flag 462, such as a sub_pic_cpb_params_present_flag element. Thesub-picture CPB parameters present flag 462 can indicate if sub-pictureCPB parameters are present in the video bitstream 110.

The HEVC VUI syntax 402 can include the subunit ticks 464, such as anum_of_units_in_sub_tick element. The subunit ticks 464 can indicate thenumber of ticks to wait before removing timing supplemental enhancementinformation (SEI) messages.

The HEVC VUI syntax 402 can include a bitstream restriction flag 466,such as a bitstream_restriction_flag element. The bitstream restrictionflag 466 can indicate that the coded video sequence bitstreamrestriction parameters are present in the video bitstream 110. If thebitstream restriction flag 466 has a value of 1 the HEVC VUI syntax 402can include a tiles fixed structure flag 468, a motion vector flag 470,a max bytes per picture denomination 472, a maximum bits per minimum cudenomination 474, a maximum motion vector horizontal length 476, and amaximum motion vector vertical length 478.

The HEVC VUI syntax 402 can include the tiles fixed structure flag 468,such as a tiles_fixed_structure_flag element, can indicate that eachpicture in the coded video sequence has the same number of tiles. Thetiles fixed structure flag 468 can have to value of 1 to indicate thatfixed tiles and a value of 0 to indicate otherwise.

The HEVC VUI syntax 402 can include the motion vector flag 470, such asa motion_vector_over_pic_boundaries_flag element, can indicate that nosample outside the picture boundaries is used for prediction. If themotion vector flag 470 has a value of 1, then one or more samplesoutside the picture boundaries may be used for prediction, otherwise nosamples are used for prediction.

The HEVC VUI syntax 402 can include the max bytes per picturedenomination 472, such as a max_bytes_per_pic_denom element, is a valueindicating the maximum number of bytes for the sum of the sizes of theVCL NAL units associated with any coded picture in the coded videosequence. If the max bytes per picture denomination 472 has a value of0, then no limits are indicated. Otherwise, it is a requirement ofbitstream conformance that no coded pictures shall be represented in thevideo bitstream 110 by more bytes than the max bytes per picturedenomination 472.

The HEVC VUI syntax 402 can include the maximum bits per minimum cudenomination 474, such as a max_bits_per_min cu_denom element, is avalue indicating the an upper bound for the number of coded bits ofcoding unit data for any coding block in any picture of the coded videosequence. If the maximum bits per minimum cu denomination 474 has avalue of 0, then no limit is indicated. Otherwise, is a requirement ofbitstream conformance that no coding unit shall be represented in thebitstream by more than the maximum bits per minimum cu denomination 474.

The HEVC VUI syntax 402 can include the maximum motion vector horizontallength 476, such as a log2_max_mv_length_horizontal element, indicatesthe maximum absolute value of a decoded horizontal motion vectorcomponent for all pictures in the video bitstream 110. The maximummotion vector vertical length 478, such as a log2_max_mv_length_verticalelement, indicates the maximum absolute value of a decoded verticalmotion vector component for all pictures in the video bitstream 110.

The HRD syntax 302 can represent a set of normative requirements for thevideo bitstream 110. The HRD syntax 302 can be used to control the bitrate of the video bitstream 110. For example, the HRD syntax 302 caninclude parameters for controlling variable or constant bit rateoperations, low-delay behavior, and delay-tolerant behavior.

In another example, the HRD syntax 302 be used to control the codedpicture buffer performance, the number of coded picture buffers, and thesize of the coded picture buffers using parameters such as the bit ratescale 310 of FIG. 3, the CPB count 308 of FIG. 3, and the CPB size scale312 of FIG. 3. The HRD syntax 302 can be used for controlling thedecoded picture buffer using parameters such as the DPB output delaylength 326 of FIG. 3.

It has been discovered that using the HRD syntax 302 provides improvedperformance by enabling finer grained control over the processing of theindividual occurrences of the coded picture buffer. Using individualoccurrences of the HRD syntax 302 can provide improved processing speedby taking advantage of individual differences between differentoccurrences of the CPB.

It has been discovered that encoding and decoding the video content 108using the HRD syntax 302 can reduce the size of the video bitstream 110and reduces the amount of video buffering required to display the videostream 112. Reducing the size of the video bitstream 110 increasesfunctionality and increases the performance of display of the videostream 112.

Referring now to FIG. 5, therein is shown an example of a HEVC VUIextension syntax 502. The HEVC VUI extension syntax 502 providesinformation for each occurrence of the temporal layers in the videobitstream 110 of FIG. 1. The HEVC VUI extension syntax 502 can be anembodiment of the HEVC VUI syntax 402 of FIG. 4.

The HEVC VUI extension syntax 502 describes the elements in the HEVC VUIextension syntax table of FIG. 5. The elements of the HEVC VUI extensionsyntax 502 are arranged in a hierarchical structure as described in theHEVC VUI extension syntax table of FIG. 5.

The HEVC VUI syntax 402 can describe the VUI parameters of the videocoding system 100 of FIG. 1. For example, the HEVC VUI extension syntax502 can be an occurrence of the HEVC VUI syntax 402. Terms such as firstor second are used for identification purposes only and do not indicateany order, priority, importance, or precedence.

The HEVC VUI extension syntax 502 includes a HEVC VUI extension syntaxheader 504, such as the vui_parameters element. The HEVC VUI extensionsyntax header 504 is a descriptor for identifying the HEVC VUI extensionsyntax 502.

The HEVC VUI extension syntax 502 can include the NAL HRD parameterspresent flag 452, such as the nal_hrd_parameters_present_flag element.The NAL HRD parameters present flag 452 can indicate the presence of theNAL HRD parameter information.

The HEVC VUI extension syntax 502 can include the VCL HRD parameterspresent flag 456, such as the vcl_hrd_parameters_present_flag element.The VCL HRD parameters present flag 456 can indicate the presence of theVCL HRD parameter information.

If the NAL HRD parameters present flag 452 or the VCL HRD parameterspresent flag 456 have a value of 1, then the HEVC VUI extension syntax502 can include the low delay flag 460 and the sub-picture CPBparameters present flag 462.

The NAL HRD parameters present flag 452 and the VCL HRD parameterspresent flag 456 can control the inclusion of other HRD-relatedparameters. If the NAL HRD parameters present flag 452 or the VCL HRDparameters present flag 456 have a value of 1, then the HEVC VUIextension syntax 502 can include the low delay flag 460 and thesub-picture CPB parameters present flag 462.

The HEVC VUI extension syntax 502 can include the low delay flag 460,such as the low delay hrd flag element. The low delay flag 460 canindicates the HRD operational mode.

The HEVC VUI extension syntax 502 can include the sub-picture CPBparameters present flag 462, such as the sub_pic_cpb_params_present_flagelement. The sub-picture CPB parameters present flag 462 can indicate ifsub-picture CPB parameters are present in the video bitstream 110 ofFIG. 1.

If the sub-picture CPB parameters present flag 462 has a value of 1,then the HEVC VUI extension syntax 502 can include the subunit ticks464, such as the num_of_units_in_sub_tick element. The subunit ticks 464can indicate the number of ticks to wait before removing timingsupplemental enhancement information (SEI) messages.

The HEVC VUI extension syntax 502 can include two conditional checks tobe evaluated to determine if the HRD syntax 302 of FIG. 3 is included inthe HEVC VUI extension syntax 502. If the NAL HRD parameters presentflag 452 has a value of 1, then the HEVC VUI extension syntax 502 caninclude the HRD syntax 302.

If the VCL HRD parameters present flag 456 has a value of 1, then theHEVC VUI syntax 402 can include the HRD syntax 302. If neither the NALHRD parameters present flag 452 or the VCL HRD parameters present flag456 has a value of 1, then the HRD syntax 302 is not included in theHEVC VUI syntax 402.

The video bitstream 110 can include an occurrence of the HEVC VUIextension syntax 502. The video bitstream 110 is a serial bitstream witheach element of the HEVC VUI extension syntax 502 ordered sequentiallywithin the video bitstream 110. The elements of the HEVC VUI extensionsyntax 502 can be extracted from the video bitstream 110 in the orderdefined in the HEVC VUI extension syntax table of FIG. 5.

The value of the low delay flag 460 of the HEVC VUI extension syntax 502can determine the usage of the CPB count 308 of FIG. 3 of the HRD syntax302. If the low delay flag 460 has a value of 1, then the CPB count 308is set to 0.

The HEVC VUI extension syntax 502 includes the low delay flag 460positioned before the HRD syntax 302 in the serial transmission of thevideo bitstream 110. The low delay flag 460 is extracted before the HRDsyntax 302. The NAL HRD parameters present flag 452 and VCL HRDparameters present flag 456 are extracted before the HRD syntax 302. Theelements of the HRD syntax 302 can be extracted based on the value ofthe low delay flag 460, the NAL HRD parameters present flag 452, and theVCL HRD parameters present flag 456. For example, if the low delay flag460 has a value of 1 and either the NAL HRD parameters present flag 452or the VCL HRD parameters present flag 456 has a value of 1, then thevalue of the CPB count 308 of the HRD syntax 302 can be expressly set to0 and the video coding system 100 can operating in a low delay mode withonly a single coded picture buffer.

It has been discovered that encoding and decoding the video content 108of FIG. 1 using the HEVC VUI extension syntax 502 having the HRDparameters structure 454 constant for all of the temporal layers 230 ofFIG. 2 provides reduced complexity and increased performance. The HRDparameters structure 454 provides simplified performance and reducedcomplexity by enabling consistent control over the processing of thedecoding process.

Referring now to FIG. 6, therein is shown an example of a HRD basesyntax 602. The HRD base syntax 602 describes the parameters associatedwith the hypothetical reference decoder operation.

The HRD base syntax 602 includes elements as described in the HRD syntaxtable of FIG. 6. The elements of the HRD base syntax 602 are arranged ina hierarchical structure as described in the HRD syntax table of FIG. 6.

The HRD base syntax 602 can include a HRD base syntax header 604, suchas the hrd_parameters element. The HRD base syntax header 604 is adescriptor for identifying the HRD base syntax 602.

The HRD base syntax 602 can include the timing information present flag442, such as the timing_info_present_flag element, to indicate whethertiming information is included in the video bitstream 110 of FIG. 1. Thetiming information present flag 442 can have a value of 1 to indicatetiming information is in the video bitstream 110 and a value of 0 toindicate that timing information is not included in the video bitstream110.

The HRD base syntax 602 can include the tick units 444, such as thenum_units_in_tick element, to indicate the number of time units of aclock operating at the frequency of the time scale 446. For example, thetick units 444 can have corresponding to the minimum interval of timethat can be represented in the video bitstream 110. The time scale 446,such as the time_scale element, is the number of time units that pass inone second.

The HRD base syntax 602 can include the NAL HRD parameters present flag452, such as the nal_hrd_parameters_present_flag element, to indicatethe presence of the NAL HRD parameter information. The HRD base syntax602 can include the VCL HRD parameters present flag 456, such as thevcl_hrd_parameters_present_flag element, to indicate the presence of theHRD information for VCL.

If the NAL HRD parameters present flag 452 or the VCL HRD parameterspresent flag 456 has a value of 1, then the HRD base syntax 602 caninclude additional CPB-related elements. For example, the HRD basesyntax 602 can include the sub-picture CPB parameters present flag 462,the bit rate scale 310, the CPB size scale 312, the initial CPB removaldelay length 322, the CPB removal delay length 324, and the DPB outputdelay length 326.

The HRD base syntax 602 can include the sub-picture CPB parameterspresent flag 462, such as the sub_pic_cpb_params_present_flag element,to indicate if sub-picture CPB parameters are present in the videobitstream 110. If the sub-picture CPB parameters present flag 462 has avalue of 1, then the HRD base syntax 602 can include a tick divisor 606,such as a tick_divisor_minus2 element, to specify the minimum intervalof time that can be represented in the video bitstream 110.

The HRD base syntax 602 can include the bit rate scale 310, such as abit_rate_scale element, to indicate the maximum input bit rate of codedpicture buffer (CPB). The HRD base syntax 602 can include the CPB sizescale 312, such as a cpb_size_scale element, for determining the size ofthe CPB.

The HRD base syntax 602 can include the initial CPB removal delay length322, such as an initial_cpb_removal_delay_length_minus1 element, toindicate the bit length of the elements of the buffering period SEImessage. The HRD base syntax 602 can include the CPB removal delaylength 324, such as a cpb_removal_delay_length_minus1 element, toindicate the bit length of the elements cpb_removalv delay in thepicture timing SEI message.

The HRD base syntax 602 can include the DPB output delay length 326,such as a dpb_output_delay_length_minus1 element. The DPB output delaylength 326 indicates the size of the decoded picture buffer (DPB).

The HRD base syntax 602 can include a set of parameters for eachoccurrence of the sub-layers. The HRD base syntax 602 can include a loopstructure using an iterator, such as [i], to describe parameters foreach occurrence of the sub-layer.

The HRD base syntax 602 can include a sub-layer count 306, such as theMaxNumSubLayersMinus1 element, to indicate the maximum number of thesub-layers in the video bitstream 110. The HRD base syntax 602 caninclude the fixed picture rate flag 448, such as a fixed_pic_rate_flagelement, to indicate whether the temporal distance between the HRDoutput times of any two consecutive pictures in the video bitstream 110is constrained.

If the fixed picture rate flag 448 has a value of 1, then the HRD basesyntax 602 can include a picture duration 608, such as apic_duration_in_tc_minus1 element. The picture duration 608 can indicatethe temporal distance between the HRD output times of any twoconsecutive pictures in output order in the coded video sequence.

The HRD base syntax 602 can include the low delay flag 460, such as alow_delay_hrd_flag element. The low delay flag 460 can indicate the HRDoperational mode.

The HRD base syntax 602 can include the CPB count 308, such as acpb_cnt_minus1 element. The CPB count 308 can indicate the number ofalternative CPB specification in the video bitstream 110.

If the NAL HRD parameters present flag 452 has a value of 1, then theHRD base syntax 602 can include a HRD sub-layers parameters structure610, such as a hrd_parameters_sub_layer element, for each occurrence ofthe sub-layers. The HRD sub-layers parameters structure 610 can describethe parameters related to each sub-layer.

If the VCL HRD parameters present flag 456 has a value of 1, then theHRD base syntax 602 can include the HRD sub-layers parameters structure610, such as a hrd_parameters_sub_layer element, for each occurrence ofthe temporal layers 230 of FIG. 2. The HRD sub-layers parametersstructure 610 can describe the parameters related to each sub-layer.

The HRD base syntax 602 includes the low delay flag 460 positionedbefore the HRD sub-layers parameters structure 610 in the serialtransmission of the video bitstream 110. The low delay flag 460 isextracted before the HRD sub-layers parameters structure 610. The NALHRD parameters present flag 452 and VCL HRD parameters present flag 456are extracted before the HRD sub-layers parameters structure 610.

The elements of the HRD sub-layers parameters structure 610 can beextracted based on the value of the low delay flag 460, the NAL HRDparameters present flag 452, and the VCL HRD parameters present flag456. For example, if the low delay flag 460 has a value of 1 and eitherthe NAL HRD parameters present flag 452 or the VCL HRD parameterspresent flag 456 has a value of 1, then the value of the CPB count 308of the HRD sub-layers parameters structure 610 can be expressly set to 0and the video coding system 100 can operating in a low delay mode withonly a single coded picture buffer.

It has been discovered that encoding and decoding the video content 108of FIG. 1 using the HRD base syntax 602 can reduce the size of the videobitstream 110 and reduces the amount of video buffering required todisplay the video stream 112 of FIG. 1. Reducing the size of the videobitstream 110 increases functionality and increases the performance ofdisplay of the video stream 112.

Referring now to FIG. 7, therein is shown an example of a HRD sub-layersyntax 702. The HRD sub-layer syntax 702 describes the parametersassociated with the sub-layers of the temporal layers for thehypothetical reference decoder.

The HRD sub-layer syntax 702 includes elements as described in the HRDsub-layer syntax table of FIG. 7. The elements of the HRD sub-layersyntax 702 are arranged in a hierarchical structure as described in theHRD sub-layer syntax table of FIG. 7.

The HRD sub-layer syntax 702 can include a HRD sub-layer syntax header704, such as a HRD_parameters_sub_layer element. The HRD sub-layersyntax header 704 is a descriptor for identifying the HRD sub-layersyntax 702.

The HRD sub-layer syntax 702 can include a loop structure to define aset of parameters for each occurrence of the coded picture buffer. Theloop structure is dimensioned based on the schedule selection index 314,such as a SchedSelIdx element.

The HRD sub-layer syntax 702 can include the bit rate value 316, such asa bit_rate_value_minus1 element. The bit rate value 316 can be used tospecify the maximum input bit rate for each occurrence of the codedpicture buffer.

The HRD sub-layer syntax 702 can include the CPB size value 318, such asa cpb_size_value_minus1 element. The CPB size value 318 can be used todetermine the size of each occurrence of the coded picture buffer.

The HRD sub-layer syntax 702 can include the CBR flag 320, such as acbr_flag element. The CBR flag 320 indicates the operation mode fordecoding the video bitstream 110 of FIG. 1 for each occurrence of thecoded picture buffer. If the CBR flag 320 has a value of 1, then thehypothetical stream delivery schedule operates in a constant bit ratemode. Otherwise, the video bitstream 110 includes an intermittent bitrate mode.

The HRD sub-layer syntax 702 can describe properties of the temporallayers 230 of FIG. 2. The temporal layers 230 can also be designated assub-layers of the video bitstream 110 of FIG. 1.

The HRD sub-layer syntax 702 can represent the sub-layers or thetemporal layers 230 of the video bitstream 110. The HRD sub-layer syntax702 can be used to select one of the sub-layers or one of the temporallayers 230 and allow the removal occurrences of other sub-layers fromthe video bitstream 110.

Removing occurrences of the sub-layers or the temporal layers 230 canreduce the overall volume of data within the video bitstream 110 andenable bit-rate reduction or resizing of the video content 108 of FIG. 1for better transmission, improved storage bandwidth control andadjustment. Providing sub-layer or temporal layer specific HRDparameters enable better and smoother bitstream decoding to generate thevideo stream 112 of FIG. 1.

It has been discovered that using the HRD sub-layer syntax 702 providesimproved performance by enabling finer grained control over theprocessing of the individual sub-layers associated with the temporallayers 230 of FIG. 2. Using individual occurrences of the HRD sub-layersyntax 702 can provide improved processing speed by taking advantage ofindividual differences between different sub-layers.

Referring now to FIG. 8, therein is shown an example of a HRD VUI syntax802. The HRD VUI syntax 802 describes the parameters associated with thehypothetical reference decoder.

The HRD VUI syntax 802 includes elements as described in the HRD VUIsyntax table of FIG. 8. The elements of the HRD VUI syntax 802 arearranged in a hierarchical structure as described in the HRD VUI syntaxtable of FIG. 8.

The HRD VUI syntax 802 can include a HRD VUI syntax header 804, such asthe vui_parameters element. The HRD VUI syntax header 804 is adescriptor for identifying the HRD VUI syntax 802.

The HRD VUI syntax 802 can include the aspect ratio flag 406, such asthe aspect_ratio_info_present_flag element, to show that additionalaspect ratio information is encoded in the video bitstream 110 ofFIG. 1. The HRD VUI syntax 802 can include the aspect ratio indicator408, such as the aspect_ratio_idc element, to describe the aspect ratioof the video content 108 of FIG. 1.

The aspect ratio indicator 408 can include a value indicating that theaspect ratio can be described by individual values for the aspect ratiowidth 410 and the aspect ratio height 412. The aspect ratio width 410,such as the sar_width element, can describe the width of the videocontent 108. The aspect ratio height 412, such as the sar_heightelement, can describe the height of the video content 108.

The HRD VUI syntax 802 can include the overscan present flag 414, suchas the overscan_info_present_flag element, to indicate if overscaninformation is included in the video bitstream 110. If the overscanpresent flag 414 has a value of 1, then the HRD VUI syntax 802 caninclude the overscan appropriate flag 416, such as anoverscan_appropriate_flag element, to indicate that the video content108 encoded in the video bitstream 110 can be displayed using overscan.

The HRD VUI syntax 802 can include the video signal present flag 418,such as the video_signal_type_present_flag element, to indicate thatvideo signal type information is included in the video bitstream 110. Ifthe video signal present flag 418 has a value of 1, the HRD VUI syntax802 can include the video format 420, the video full range flag 422, andthe color description present flag 424.

The video format 420, such as the video_format element, can indicate theformat of the video. The video full range flag 422, such as thevideo_full_range_flag element, can indicate the black level and therange of the luma and chroma signals for the video content 108 encodedin the video bitstream 110.

The color description present flag 424, such as thecolour_description_present_flag element, can indicate the presence ofcolor description information in the video bitstream 110. The colordescription information can include the color primaries 426, thetransfer characteristics 428, and the matrix coefficient 430.

The color primaries 426, such as the colour_primaries element, canindicate the color scheme used in the video content 108. The transfercharacteristics 428 can indicate the opto-electronic transfercharacteristics of the video content 108. The matrix coefficient 430,such as the matrix_coefficient element, can indicate coefficient used toderive luma and chroma signals from the red, green, and blue primariesindicated by the color primaries 426.

The HRD VUI syntax 802 can include the chroma location informationpresent flag 432, such as the chroma_loc_info_present_flag element, toindicate whether additional chroma information is present in the videobitstream 110. If the chroma location information present flag 432 canhas a value of 1, then the HRD VUI syntax 802 can include the chroma topfield sample 434 and the chroma bottom field sample 436.

The chroma top field sample 434, such as the chroma sample loc type topfield element, can be an enumerated value to specify the location ofchroma samples for the top field in the video bitstream 110. The chromabottom field sample 436, such as the chroma_sample_loc_type_bottom_fieldelement, can be an enumerated value to specify the location of chromasamples for the bottom field in the video bitstream 110.

The HRD VUI syntax 802 can include the neutral chroma flag 438, such asthe neutral_chroma_indication_flag element, can indicate whether thedecoded chroma samples are equal to one. The HRD VUI syntax 802 caninclude the field sequence flag 440, such as the field_seq_flag, toindicate whether coded video sequence information includes videorepresenting fields.

The HRD VUI syntax 802 can include the HRD parameters structure 454,such as the hrd_parameters element. The HRD parameters structure 454 caninclude the hypothetical reference decoder parameters for eachsub-layer.

The HRD VUI syntax 802 can include the bitstream restriction flag 466,such as a bitstream_restriction_flag element, to indicate that the codedvideo sequence bitstream restriction parameters are present in the videobitstream 110. If the bitstream restriction flag 466 has a value of 1the HRD VUI syntax 802 can include the tiles fixed structure flag 468,the motion vector flag 470, the max bytes per picture denomination 472,the maximum bits per minimum cu denomination 474, the maximum motionvector horizontal length 476, and the maximum motion vector verticallength 478.

The HRD VUI syntax 802 can include the tiles fixed structure flag 468,such as a tiles_fixed_structure_flag element, to indicate that eachpicture in the coded video sequence has the same number of tiles. TheHRD VUI syntax 802 can include the motion vector flag 470, such as amotion_vector_over_pic_boundaries_flag element, to indicate that nosample outside the picture boundaries is used for prediction.

The HRD VUI syntax 802 can include the max bytes per picturedenomination 472, such as a max_bytes_per_pic_denom element, to indicatethe maximum number of bytes for the sum of the sizes of the VCL NALunits associated with any coded picture in the coded video sequence. TheHRD VUI syntax 802 can include the maximum bits per minimum cudenomination 474, such as a max_bits_per_min_cu_denom element, toindicate the upper bound for the number of coded bits of coding unitdata for any coding block in any picture of the coded video sequence.

The HRD VUI syntax 802 can include the maximum motion vector horizontallength 476, such as a log2_max_mv_length_horizontal element, to indicatethe maximum absolute value of a decoded horizontal motion vectorcomponent for all pictures in the video bitstream 110. The HRD VUIsyntax 802 can include the maximum motion vector vertical length 478,such as a log2_max_mv_length_vertical element, to indicate the maximumabsolute value of a decoded vertical motion vector component for allpictures in the video bitstream 110.

It has been discovered that using the HRD parameters structure 454 inthe HRD VUI syntax 802 provides improved performance by enabling finergrained control over the processing of the individual sub-layers. Usingindividual occurrences of the HRD parameters structure 454 can provideimproved processing speed by taking advantage of individual differencesbetween different sub-layers.

Referring now to FIG. 9, therein is shown a functional block diagram ofthe video coding system 100. The video coding system 100 can include thefirst device 102, the second device 104 and the communication path 106.

The first device 102 can communicate with the second device 104 over thecommunication path 106. The first device 102 can send information in afirst device transmission 932 over the communication path 106 to thesecond device 104. The second device 104 can send information in asecond device transmission 934 over the communication path 106 to thefirst device 102.

For illustrative purposes, the video coding system 100 is shown with thefirst device 102 as a client device, although it is understood that thevideo coding system 100 can have the first device 102 as a differenttype of device. For example, the first device can be a server. In afurther example, the first device 102 can be the video encoder 102, thevideo decoder 104, or a combination thereof.

Also for illustrative purposes, the video coding system 100 is shownwith the second device 104 as a server, although it is understood thatthe video coding system 100 can have the second device 104 as adifferent type of device. For example, the second device 104 can be aclient device. In a further example, the second device 104 can be thevideo encoder 102, the video decoder 104, or a combination thereof.

For brevity of description in this embodiment of the present invention,the first device 102 will be described as a client device, such as avideo camera, smart phone, or a combination thereof. The presentinvention is not limited to this selection for the type of devices. Theselection is an example of the present invention.

The first device 102 can include a first control unit 908. The firstcontrol unit 908 can include a first control interface 914. The firstcontrol unit 908 can execute a first software 912 to provide theintelligence of the video coding system 100.

The first control unit 908 can be implemented in a number of differentmanners. For example, the first control unit 908 can be a processor, anembedded processor, a microprocessor, a hardware control logic, ahardware finite state machine (FSM), a digital signal processor (DSP),or a combination thereof.

The first control interface 914 can be used for communication betweenthe first control unit 908 and other functional units in the firstdevice 102. The first control interface 914 can also be used forcommunication that is external to the first device 102.

The first control interface 914 can receive information from the otherfunctional units or from external sources, or can transmit informationto the other functional units or to external destinations. The externalsources and the external destinations refer to sources and destinationsexternal to the first device 102.

The first control interface 914 can be implemented in different ways andcan include different implementations depending on which functionalunits or external units are being interfaced with the first controlinterface 914. For example, the first control interface 914 can beimplemented with electrical circuitry, microelectromechanical systems(MEMS), optical circuitry, wireless circuitry, wireline circuitry, or acombination thereof.

The first device 102 can include a first storage unit 904. The firststorage unit 904 can store the first software 912. The first storageunit 904 can also store the relevant information, such as images, syntaxinformation, video, maps, profiles, display preferences, sensor data, orany combination thereof.

The first storage unit 904 can be a volatile memory, a nonvolatilememory, an internal memory, an external memory, or a combinationthereof. For example, the first storage unit 904 can be a nonvolatilestorage such as non-volatile random access memory (NVRAM), Flash memory,disk storage, or a volatile storage such as static random access memory(SRAM).

The first storage unit 904 can include a first storage interface 918.The first storage interface 918 can be used for communication betweenthe first storage unit 904 and other functional units in the firstdevice 102. The first storage interface 918 can also be used forcommunication that is external to the first device 102.

The first device 102 can include a first imaging unit 906. The firstimaging unit 906 can capture the video content 108 of FIG. 1 from thereal world. The first imaging unit 906 can include a digital camera, anvideo camera, an optical sensor, or any combination thereof.

The first imaging unit 906 can include a first imaging interface 916.The first imaging interface 916 can be used for communication betweenthe first imaging unit 906 and other functional units in the firstdevice 102.

The first imaging interface 916 can receive information from the otherfunctional units or from external sources, or can transmit informationto the other functional units or to external destinations. The externalsources and the external destinations refer to sources and destinationsexternal to the first device 102.

The first imaging interface 916 can include different implementationsdepending on which functional units or external units are beinginterfaced with the first imaging unit 906. The first imaging interface916 can be implemented with technologies and techniques similar to theimplementation of the first control interface 914.

The first storage interface 918 can receive information from the otherfunctional units or from external sources, or can transmit informationto the other functional units or to external destinations. The externalsources and the external destinations refer to sources and destinationsexternal to the first device 102.

The first storage interface 918 can include different implementationsdepending on which functional units or external units are beinginterfaced with the first storage unit 904. The first storage interface918 can be implemented with technologies and techniques similar to theimplementation of the first control interface 914.

The first device 102 can include a first communication unit 910. Thefirst communication unit 910 can be for enabling external communicationto and from the first device 102. For example, the first communicationunit 910 can permit the first device 102 to communicate with the seconddevice 104, an attachment, such as a peripheral device or a computerdesktop, and the communication path 106.

The first communication unit 910 can also function as a communicationhub allowing the first device 102 to function as part of thecommunication path 106 and not limited to be an end point or terminalunit to the communication path 106. The first communication unit 910 caninclude active and passive components, such as microelectronics or anantenna, for interaction with the communication path 106.

The first communication unit 910 can include a first communicationinterface 920. The first communication interface 920 can be used forcommunication between the first communication unit 910 and otherfunctional units in the first device 102. The first communicationinterface 920 can receive information from the other functional units orcan transmit information to the other functional units.

The first communication interface 920 can include differentimplementations depending on which functional units are being interfacedwith the first communication unit 910. The first communication interface920 can be implemented with technologies and techniques similar to theimplementation of the first control interface 914.

The first device 102 can include a first user interface 902. The firstuser interface 902 allows a user (not shown) to interface and interactwith the first device 102. The first user interface 902 can include afirst user input (not shown). The first user input can include touchscreen, gestures, motion detection, buttons, sliders, knobs, virtualbuttons, voice recognition controls, or any combination thereof.

The first user interface 902 can include the first display interface120. The first display interface 120 can allow the user to interact withthe first user interface 902. The first display interface 120 caninclude a display, a video screen, a speaker, or any combinationthereof.

The first control unit 908 can operate with the first user interface 902to display video information generated by the video coding system 100 onthe first display interface 120. The first control unit 908 can alsoexecute the first software 912 for the other functions of the videocoding system 100, including receiving video information from the firststorage unit 904 for display on the first display interface 120. Thefirst control unit 908 can further execute the first software 912 forinteraction with the communication path 106 via the first communicationunit 910.

For illustrative purposes, the first device 102 can be partitionedhaving the first user interface 902, the first storage unit 904, thefirst control unit 908, and the first communication unit 910, althoughit is understood that the first device 102 can have a differentpartition. For example, the first software 912 can be partitioneddifferently such that some or all of its function can be in the firstcontrol unit 908 and the first communication unit 910. Also, the firstdevice 102 can include other functional units not shown in FIG. 1 forclarity.

The video coding system 100 can include the second device 104. Thesecond device 104 can be optimized for implementing the presentinvention in a multiple device embodiment with the first device 102. Thesecond device 104 can provide the additional or higher performanceprocessing power compared to the first device 102.

The second device 104 can include a second control unit 948. The secondcontrol unit 948 can include a second control interface 954. The secondcontrol unit 948 can execute a second software 952 to provide theintelligence of the video coding system 100.

The second control unit 948 can be implemented in a number of differentmanners. For example, the second control unit 948 can be a processor, anembedded processor, a microprocessor, a hardware control logic, ahardware finite state machine (FSM), a digital signal processor (DSP),or a combination thereof.

The second control interface 954 can be used for communication betweenthe second control unit 948 and other functional units in the seconddevice 104. The second control interface 954 can also be used forcommunication that is external to the second device 104.

The second control interface 954 can receive information from the otherfunctional units or from external sources, or can transmit informationto the other functional units or to external destinations. The externalsources and the external destinations refer to sources and destinationsexternal to the second device 104.

The second control interface 954 can be implemented in different waysand can include different implementations depending on which functionalunits or external units are being interfaced with the second controlinterface 954. For example, the second control interface 954 can beimplemented with electrical circuitry, microelectromechanical systems(MEMS), optical circuitry, wireless circuitry, wireline circuitry, or acombination thereof.

The second device 104 can include a second storage unit 944. The secondstorage unit 944 can store the second software 952. The second storageunit 944 can also store the relevant information, such as images, syntaxinformation, video, maps, profiles, display preferences, sensor data, orany combination thereof.

The second storage unit 944 can be a volatile memory, a nonvolatilememory, an internal memory, an external memory, or a combinationthereof. For example, the second storage unit 944 can be a nonvolatilestorage such as non-volatile random access memory (NVRAM), Flash memory,disk storage, or a volatile storage such as static random access memory(SRAM).

The second storage unit 944 can include a second storage interface 958.The second storage interface 958 can be used for communication betweenthe second storage unit 944 and other functional units in the seconddevice 104. The second storage interface 958 can also be used forcommunication that is external to the second device 104.

The second storage interface 958 can receive information from the otherfunctional units or from external sources, or can transmit informationto the other functional units or to external destinations. The externalsources and the external destinations refer to sources and destinationsexternal to the second device 104.

The second storage interface 958 can include different implementationsdepending on which functional units or external units are beinginterfaced with the second storage unit 944. The second storageinterface 958 can be implemented with technologies and techniquessimilar to the implementation of the second control interface 954.

The second device 104 can include a second imaging unit 946. The secondimaging unit 946 can capture the video content 108 from the real world.The first imaging unit 906 can include a digital camera, an videocamera, an optical sensor, or any combination thereof.

The second imaging unit 946 can include a second imaging interface 956.The second imaging interface 956 can be used for communication betweenthe second imaging unit 946 and other functional units in the seconddevice 104.

The second imaging interface 956 can receive information from the otherfunctional units or from external sources, or can transmit informationto the other functional units or to external destinations. The externalsources and the external destinations refer to sources and destinationsexternal to the second device 104.

The second imaging interface 956 can include different implementationsdepending on which functional units or external units are beinginterfaced with the second imaging unit 946. The second imaginginterface 956 can be implemented with technologies and techniquessimilar to the implementation of the first control interface 914.

The second device 104 can include a second communication unit 950. Thesecond communication unit 950 can enable external communication to andfrom the second device 104. For example, the second communication unit950 can permit the second device 104 to communicate with the firstdevice 102, an attachment, such as a peripheral device or a computerdesktop, and the communication path 106.

The second communication unit 950 can also function as a communicationhub allowing the second device 104 to function as part of thecommunication path 106 and not limited to be an end point or terminalunit to the communication path 106. The second communication unit 950can include active and passive components, such as microelectronics oran antenna, for interaction with the communication path 106.

The second communication unit 950 can include a second communicationinterface 960. The second communication interface 960 can be used forcommunication between the second communication unit 950 and otherfunctional units in the second device 104. The second communicationinterface 960 can receive information from the other functional units orcan transmit information to the other functional units.

The second communication interface 960 can include differentimplementations depending on which functional units are being interfacedwith the second communication unit 950. The second communicationinterface 960 can be implemented with technologies and techniquessimilar to the implementation of the second control interface 954.

The second device 104 can include a second user interface 942. Thesecond user interface 942 allows a user (not shown) to interface andinteract with the second device 104. The second user interface 942 caninclude a second user input (not shown). The second user input caninclude touch screen, gestures, motion detection, buttons, sliders,knobs, virtual buttons, voice recognition controls, or any combinationthereof.

The second user interface 942 can include a second display interface943. The second display interface 943 can allow the user to interactwith the second user interface 942. The second display interface 943 caninclude a display, a video screen, a speaker, or any combinationthereof.

The second control unit 948 can operate with the second user interface942 to display information generated by the video coding system 100 onthe second display interface 943. The second control unit 948 can alsoexecute the second software 952 for the other functions of the videocoding system 100, including receiving display information from thesecond storage unit 944 for display on the second display interface 943.The second control unit 948 can further execute the second software 952for interaction with the communication path 106 via the secondcommunication unit 950.

For illustrative purposes, the second device 104 can be partitionedhaving the second user interface 942, the second storage unit 944, thesecond control unit 948, and the second communication unit 950, althoughit is understood that the second device 104 can have a differentpartition. For example, the second software 952 can be partitioneddifferently such that some or all of its function can be in the secondcontrol unit 948 and the second communication unit 950. Also, the seconddevice 104 can include other functional units not shown in FIG. 1 forclarity.

The first communication unit 910 can couple with the communication path106 to send information to the second device 104 in the first devicetransmission 932. The second device 104 can receive information in thesecond communication unit 950 from the first device transmission 932 ofthe communication path 106.

The second communication unit 950 can couple with the communication path106 to send video information to the first device 102 in the seconddevice transmission 934. The first device 102 can receive videoinformation in the first communication unit 910 from the second devicetransmission 934 of the communication path 106. The video coding system100 can be executed by the first control unit 908, the second controlunit 948, or a combination thereof.

The functional units in the first device 102 can work individually andindependently of the other functional units. For illustrative purposes,the video coding system 100 is described by operation of the firstdevice 102. It is understood that the first device 102 can operate anyof the modules and functions of the video coding system 100. Forexample, the first device 102 can be described to operate the firstcontrol unit 908.

The functional units in the second device 104 can work individually andindependently of the other functional units. For illustrative purposes,the video coding system 100 can be described by operation of the seconddevice 104. It is understood that the second device 104 can operate anyof the modules and functions of the video coding system 100. Forexample, the second device 104 is described to operate the secondcontrol unit 948.

For illustrative purposes, the video coding system 100 is described byoperation of the first device 102 and the second device 104. It isunderstood that the first device 102 and the second device 104 canoperate any of the modules and functions of the video coding system 100.For example, the first device 102 is described to operate the firstcontrol unit 908, although it is understood that the second device 104can also operate the first control unit 908.

Referring now to FIG. 10, therein is shown a control flow 1000 of thevideo coding system 100 of FIG. 1. The control flow 1000 describesdecoding the video bitstream 110 of FIG. 1 by receiving the videobitstream 110, extracting the video syntax 114 of FIG. 1, decoding thevideo bitstream 110, and displaying the video stream 112 of FIG. 1.

The video coding system 100 can include a receive module 1002. Thereceive module 1002 can receive the video bitstream 110 encoded by thevideo encoder 102 of FIG. 1.

The video bitstream 110 can be received in a variety of ways. Forexample, the video bitstream 110 can be received from the video encoder102 of FIG. 1 as a streaming serial bitstream, a pre-encoded video file(not shown), in a digital message (not shown) over the communicationpath 106 of FIG. 1, or a combination thereof.

In an illustrative example, the video bitstream 110 can be received as aserial bitstream in a timewise manner with each element of the videosyntax 114 received sequentially. The video bitstream 110 can includethe video syntax 114 such as the HEVC VUI syntax 402 of FIG. 4, the HEVCVUI extension syntax 502 of FIG. 5, the HRD VUI syntax 802 of FIG. 8,the HRD syntax 302 of FIG. 3, the HRD base syntax 602 of FIG. 6, the HRDsub-layer syntax 702 of FIG. 7, or a combination thereof.

For example, the receive module 1002 can receive the HEVC VUI syntax 402with the HRD parameters structure 454 of FIG. 4 received before the lowdelay flag 460 of FIG. 4. Similarly, the NAL HRD parameters present flag452 of FIG. 4 can be received before the HRD parameters structure 454.If the NAL HRD parameters present flag 452 has a value of 0, then theVCL HRD parameters present flag 456 of FIG. 4 can be received before theHRD parameters structure 454.

The video bitstream 110 can include one or more the temporal layers 230of FIG. 2 for representing the video content 108 of FIG. 1 at differentframe rates. The receive module 1002 can selectively filter the temporallayers 230 to reduce the size of the video bitstream 110.

For example, the receive module 1002 can receive the video bitstream 110having the temporal layers 230 for three different frame rates, such as60 fps, 30 fps, and 15 fps. The receive module 1002 can filter the videobitstream 110 to remove the 60 fps and the 30 fps occurrences of thetemporal layers 230 and only process the 15 fps occurrence of thetemporal layers 230.

The video coding system 100 can include a get syntax module 1004. Theget syntax module 1004 can identify and extract the video syntax 114 ofthe video bitstream 110. The get syntax module 1004 can include a gettemporal layers module 1008 and a decode temporal layers module 1010.

The get syntax module 1004 can extract the video syntax 114 for thevideo bitstream 110 in a variety of ways. For example, the get syntaxmodule 1004 can extract the video syntax 114 by searching the videobitstream 110 for headers indicating the presence of the video syntax114. In another example, the video syntax 114 can be extracted from thevideo bitstream 110 using a demultiplexer (not shown) to separate thevideo syntax 114 from the video image data of the video bitstream 110.

In yet another example, the video syntax 114 can be extracted from thevideo bitstream 110 by extracting a sequence parameter set Raw ByteSequence Payload (RBSP) syntax. The sequence parameter set RBSP is asyntax structure containing an integer number of bytes encapsulated in anetwork abstraction layer unit. The RBSP can be either empty or have theform of a string of data bits containing syntax elements followed by aRBSP stop bit and followed by zero or more addition bits equal to 0.

The video syntax 114 can be extracted from a serial bitstream in atimewise manner by extracting individual elements as the elements areavailable in time order in the video bitstream 110. The video codingsystem 100 can selectively extract and process later elements based onthe values of the earlier extracted elements.

For example, the get syntax module 1004 can process the HRD syntax 302based on the previously received value of the low delay flag 460. TheHEVC VUI extension syntax 502 includes the low delay flag 460 positionedbefore the HRD syntax 302 in the serial transmission of the videobitstream 110. The low delay flag 460 is extracted before the HRD syntax302. The NAL HRD parameters present flag 452 and VCL HRD parameterspresent flag 456 are extracted before the HRD syntax 302.

The elements of the HRD syntax 302 can be extracted based on the valueof the low delay flag 460, the NAL HRD parameters present flag 452, andthe VCL HRD parameters present flag 456. For example, if the low delayflag 460 has a value of 1 and either the NAL HRD parameters present flag452 or the VCL HRD parameters present flag 456 has a value of 1, thenthe value of the CPB count 308 of the HRD syntax 302 can be extractedand expressly set to 0 by the get syntax module 1004 and the videocoding system 100 can operating in a low delay mode with a single codedpicture buffer.

In another example, if the video bitstream 110 is received in a file,then the video syntax 114 can be detected by examining the fileextension of the file containing the video bitstream 110. In yet anotherexample, if the video bitstream 110 is received as a digital messageover the communication path 106 of FIG. 1, then the video syntax 114 canbe provided as a portion of the structure of the digital message.

It has been discovered that the get syntax module 1004 can increaseperformance by dynamically decoding the video bitstream 110 to processthe HRD parameters structure 454 based on previously extractedoccurrences of the low delay flag 460. For example, receiving the lowdelay flag 460 increases decoding performance by changing the level ofdelay allowed in the coded picture buffers when applying the HRDparameters structure 454.

The get syntax module 1004 can extract the individual elements of thevideo syntax 114 based on the syntax type 202 of FIG. 2. The syntax type202 can include AVC video, SVC video, MVC video, MVD video, SSV video,or a combination thereof.

The get syntax module 1004 can extract the video syntax 114 having videousability information. The video syntax 114 can include the HEVC VUIsyntax 402, the HEVC VUI extension syntax 502, the HRD VUI syntax 802,or a combination thereof.

The get syntax module 1004 can extract the video syntax 114 havinghypothetical reference decoder information. The video syntax 114 caninclude the HRD syntax 302, the HRD base syntax 602, the HRD sub-layersyntax 702, or a combination thereof.

The video syntax 114 can have a variety of configurations. For example,the HEVC VUI syntax 402 can include one occurrence of the HRD syntax 302for all occurrences of the temporal layers 230. In another example, theget syntax module 1004 can include one occurrence of the HRD syntax 302for each occurrence of the temporal layers 230.

In an illustrative example, the HRD syntax 302 can include singleoccurrences of the CPB count 308, the bit rate scale 310 of FIG. 3, theCPB size scale 312 of FIG. 3, the initial CPB removal delay length 322of FIG. 3, the CPB removal delay length 324 of FIG. 3, the DPB outputdelay length 326 of FIG. 3, and the time offset length 328 of FIG. 3.The HRD syntax 302 can include a loop structure with occurrences foreach of the bit rate value 316 of FIG. 3, the CPB size value 318 of FIG.3, and the CBR flag 320 of FIG. 3 for each of the coded picture buffersas indicated by the CPB count 308.

The video coding system 100 can include a decode module 1006. The decodemodule 1006 can decode the video bitstream 110 using the video syntax114 to form the video stream 112. The decode module 1006 can include aget temporal layers module 1008 and a decode temporal layers module1010.

The decode module 1006 can decode the video bitstream 110 using thevideo syntax 114, such as the HEVC VUI syntax 402, the HEVC VUIextension syntax , the HRD VUI syntax 802, or a combination thereof. Thedecode module 1006 can identify and extract the temporal layers 230using the HRD syntax 302, the HRD base syntax 602, the HRD sub-layersyntax 702, or a combination thereof.

The get temporal layers module 1008 can identify the temporal layers 230to extract from the video bitstream 110 to form the video stream 112.The get temporal layers module 1008 can identify the temporal layers 230in a variety of ways.

For example, the get temporal layers module 1008 can identify thetemporal layers 230 by extracting the temporal layer count 238 of FIG. 2from the video syntax 114, such as HEVC VUI extension syntax. Thetemporal layer count 238 indicates the total number of temporal layers230 in the video bitstream 110.

The get temporal layers module 1008 can extract the temporal layers 230from the video bitstream 110 using the video syntax 114 to describe thedata type and size of the elements of the video syntax 114. The videosyntax 114 can include the hypothetical reference decoder parameterssyntaxes, such as the HRD syntax 302, the HRD base syntax 602, the HRDsub-layers syntax 702, or a combination thereof.

For example, the get temporal layers module 1008 can extract the aspectratio flag 406 of FIG. 4 as an unsigned 1 bit value in the videobitstream 110. Similarly, the aspect ratio height 412 of FIG. 4 and theaspect ratio width 410 of FIG. 4 can be extracted from the videobitstream 110 as unsigned 16 bit values as described in the HEVC VUIsyntax 402.

The get temporal layers module 1008 can extract the temporal layers 230by parsing the data in the video bitstream 110 based on the video syntax114. The video syntax 114 can define the number and configuration of thetemporal layers 230.

For example, the get temporal layers module 1008 can use the temporallayer count 238 to determine the total number of the temporal layers 230to extract from the video bitstream 110. The video format 420 of FIG. 4can be extracted from the video bitstream 110 to determine the type ofvideo system of the video content 108.

In another example, the CPB count 308 can be used to determine thenumber of coded picture buffers to be used to extract the temporallayers 230. The bit rate scale 310 can be used to determine the maximuminput bit rate for the coded picture buffers. The CPB size scale 312 canbe used to determine the size of the coded picture buffers.

In an illustrative example, the get temporal layers module 1008 canextract the first occurrence 232 of FIG. 2 and the second occurrence 234of FIG. 2 of the temporal layers 230 from the video bitstream 110 basedon the HRD syntax 302. The HRD syntax 302 can be common for all of thetemporal layers 230 in the video bitstream 110.

The decode temporal layers module 1010 can receive the temporal layers230 from the get temporal layers module 1008 and decode the temporallayers 230 to form the video stream 112. The decode temporal layersmodule 1010 can decode the temporal layers 230 in a variety of ways.

For example, the decode temporal layers module 1010 can decode thetemporal layers 230 using the HRD base syntax 602 to extract the videocoding layer information from the video bitstream 110. In anotherexample, the decode temporal layers module 1010 can decode the temporallayers 230 using the HRD sub-layer syntax 702. The decode temporallayers module 1010 can decode the temporal layers 230 and select one ofthe temporal layers 230 to form the video stream 112.

The video coding system 100 can include a display module 1012. Thedisplay module 1012 can receive the video stream 112 from the decodemodule 1006 and display the video stream 112 on the display interface120 of FIG. 1. The video stream 112 can include one or more occurrencesof the temporal layers 230

The physical transformation from the optical images of physical objectsof the video content 108 to displaying the video stream 112 on the pixelelements of the display interface 120 of FIG. 1 results in physicalchanges to the pixel elements of the display interface 120 in thephysical world, such as the change of electrical state the pixelelement, is based on the operation of the video coding system 100. Asthe changes in the physical world occurs, such as the motion of theobjects captured in the video content 108, the movement itself createsadditional information, such as the updates to the video content 108,that are converted back into changes in the pixel elements of thedisplay interface 120 for continued operation of the video coding system100.

The first software 912 of FIG. 9 of the first device 102 can include thevideo coding system 100. For example, the first software 912 can includethe receive module 1002, the get syntax module 1004, the decode module1006, and the display module 1012.

The first control unit 908 of FIG. 9 can execute the first software 912for the receive module 1002 to receive the video bitstream 110. Thefirst control unit 908 can execute the first software 912 for the getsyntax module 1004 to identify and extract the video syntax 114 from thevideo bitstream 110. The first control unit 908 can execute the firstsoftware 912 for the decode module 1006 to form the video stream 112.The first control unit 908 can execute the first software 912 for thedisplay module 1012 to display the video stream 112.

The second software 952 of FIG. 9 of the second device 104 of FIG. 1 caninclude the video coding system 100. For example, the second software952 can include the receive module 1002, the get syntax module 1004, andthe decode module 1006.

The second control unit 948 of FIG. 9 can execute the second software952 for the receive module 1002 to receive the video bitstream 110. Thesecond control unit 948 can execute the second software 952 for the getsyntax module 1004 to identify and extract the video syntax 114 from thevideo bitstream 110. The second control unit 948 can execute the secondsoftware 952 for the decode module 1006 to form the video stream 112 ofFIG. 1. The second control unit 948 can execute the second software forthe display module 1012 to display the video stream 112.

The video coding system 100 can be partitioned between the firstsoftware 912 and the second software 952. For example, the secondsoftware 952 can include the decode module 1006, and the display module1012. The second control unit 948 can execute modules partitioned on thesecond software 952 as previously described.

In an illustrative example, the video coding system 100 can include thevideo encoder 102 on the first device 102 and the video decoder 104 onthe second device 104. The video decoder 104 can include the displayprocessor 118 of FIG. 1 and the display interface 120.

The first software 912 can include the receive module 1002 and the getsyntax module 1004. Depending on the size of the first storage unit 904of FIG. 9, the first software 912 can include additional modules of thevideo coding system 100. The first control unit 908 can execute themodules partitioned on the first software 912 as previously described.

The first control unit 908 can operate the first communication unit 910of FIG. 9 to send the video bitstream 110 to the second device 104. Thefirst control unit 908 can operate the first software 912 to operate thefirst imaging unit 906 of FIG. 9. The second communication unit 950 ofFIG. 9 can send the video stream 112 to the first device 102 over thecommunication path 106.

The video coding system 100 describes the module functions or order asan example. The modules can be partitioned differently. For example, theget syntax module 1004 and the decode module 1006 can be combined. Eachof the modules can operate individually and independently of the othermodules.

Furthermore, data generated in one module can be used by another modulewithout being directly coupled to each other. For example, the decodemodule 1006 can receive the video bitstream 110 from the receive module1002.

The modules can be implemented in a variety of ways. The receive module1002, the get syntax module 1004, the decode module 1006, and thedisplay module 1012 can be implemented in as hardware accelerators (notshown) within the first control unit 908 or the second control unit 948,or can be implemented in as hardware accelerators (not shown) in thefirst device 102 or the second device 104 outside of the first controlunit 908 or the second control unit 948.

Referring now to FIG. 11, therein is shown a flow chart of a method 1100of operation of the video coding system in a further embodiment of thepresent invention. The method 1100 includes: receiving a video bitstreamas a serial bitstream in a block 1102; extracting a video syntax fromthe video bitstream; extracting a low delay flag, a network abstractionlayer (NAL) hypothetical reference decode (HRD) parameters present flag,and a video coding layer (VCL) HRD parameters present flag from thevideo syntax extracting a HRD syntax from the video bitstream based onthe low delay flag, the NAL HRD parameters present flag, and the VCL HRDparameters present flag in a block 1104; extracting a temporal layerfrom the video bitstream based on the video syntax having the HRD syntaxin a block 1106; and forming a video stream based on the temporal layerfor displaying on a device in a block 1108.

It has been discovered that the present invention thus has numerousaspects. The present invention valuably supports and services thehistorical trend of reducing costs, simplifying systems, and increasingperformance. These and other valuable aspects of the present inventionconsequently further the state of the technology to at least the nextlevel.

Thus, it has been discovered that the video coding system of the presentinvention furnishes important and heretofore unknown and unavailablesolutions, capabilities, and functional aspects for efficiently codingand decoding video content for high definition applications. Theresulting processes and configurations are straightforward,cost-effective, uncomplicated, highly versatile and effective, can besurprisingly and unobviously implemented by adapting known technologies,and are thus readily suited for efficiently and economicallymanufacturing video coding devices fully compatible with conventionalmanufacturing processes and technologies. The resulting processes andconfigurations are straightforward, cost-effective, uncomplicated,highly versatile, accurate, sensitive, and effective, and can beimplemented by adapting known components for ready, efficient, andeconomical manufacturing, application, and utilization.

While the invention has been described in conjunction with a specificbest mode, it is to be understood that many alternatives, modifications,and variations will be apparent to those skilled in the art in light ofthe aforegoing description. Accordingly, it is intended to embrace allsuch alternatives, modifications, and variations that fall within thescope of the included claims. All matters hithertofore set forth hereinor shown in the accompanying drawings are to be interpreted in anillustrative and non-limiting sense.

What is claimed is:
 1. A method of operation of a video coding systemcomprising: receiving a video bitstream as a serial bitstream;extracting a video syntax from the video bitstream; extracting a lowdelay flag, a network abstraction layer (NAL) hypothetical referencedecode (HRD) parameters present flag, and a video coding layer (VCL) HRDparameters present flag from the video syntax; extracting a HRD syntaxfrom the video bitstream based on the low delay flag, the NAL HRDparameters present flag, and the VCL HRD parameters present flag;extracting a temporal layer from the video bitstream based on the videosyntax having the HRD syntax; and forming a video stream based on thetemporal layer for displaying on a device.
 2. The method as claimed inclaim 1 wherein forming the video stream includes forming the videostream for a resolution greater than or equal to 3840 pixels by 2160pixels.
 3. The method as claimed in claim 1 wherein extracting the HRDsyntax includes extracting the HRD syntax common to all occurrences ofthe temporal layer.
 4. The method as claimed in claim 1 whereinextracting the HRD syntax includes extracting an occurrence of the HRDsyntax for each occurrence of the temporal layer.
 5. The method asclaimed in claim 1 wherein extracting the video syntax includes settingthe value of a coded picture buffer count based on the low delay flag.6. A method of operation a video coding system comprising: receiving avideo bitstream as a serial bitstream; identifying a syntax type of thevideo content from the video bitstream; extracting a video syntax fromthe video bitstream for the syntax type; extracting a low delay flag, anetwork abstraction layer (NAL) hypothetical reference decode (HRD)parameters present flag, and a video coding layer (VCL) HRD parameterspresent flag from the video syntax; extracting a HRD syntax from thevideo bitstream based on the low delay flag, the NAL HRD parameterspresent flag, and the VCL HRD parameters present flag; extracting atemporal layer from the video bitstream based on the video syntax havingthe HRD syntax; and forming a video stream based on the temporal layerfor displaying on a device.
 7. The method as claimed in claim 6 whereinforming the video stream includes forming the video stream for aresolution greater than or equal to 3840 pixels by 2160 pixels.
 8. Themethod as claimed in claim 6 wherein extracting the HRD syntax includesextracting the HRD syntax common to all occurrences of the temporallayer.
 9. The method as claimed in claim 6 wherein extracting the HRDsyntax includes extracting an occurrence of the HRD syntax for eachoccurrence of the temporal layer.
 10. The method as claimed in claim 6wherein extracting the video syntax includes setting the value of acoded picture buffer count based on the low delay flag.
 11. A videocoding system comprising: a receive module for receiving a videobitstream as a serial bitstream; a get syntax module, coupled to thereceive module, for extracting a video syntax from the video bitstream,extracting a low delay flag and a network abstraction layer (NAL)hypothetical reference decode (HRD) parameters present flag and a videocoding layer (VCL) HRD parameters present flag from the video syntax,and extracting a HRD syntax from the video bitstream based on the lowdelay flag, the NAL HRD parameters present flag, and the VCL HRDparameters present flag; a decode module, coupled to the get syntaxmodule, for extracting a temporal layer from the video bitstream basedon the video syntax having the HRD syntax; and a display module, coupledto the decode module, for forming a video stream based on the temporallayer for displaying on a device.
 12. The system as claimed in claim 11wherein the decode module is for forming the video stream for aresolution greater than or equal to 3840 pixels by 2160 pixels.
 13. Thesystem as claimed in claim 11 wherein the decode module is forextracting the HRD syntax common to all occurrences of the temporallayer.
 14. The system as claimed in claim 11 wherein the decode moduleis for setting the value of a coded picture buffer count based on thelow delay flag.
 15. The system as claimed in claim 11 wherein the getsyntax module is for extracting a High Efficiency Video Coding VideoUsability Information syntax.
 16. The system as claimed in claim 11wherein: the get syntax module is for identifying a syntax type of thevideo content from the video bitstream; and the get syntax module is forextracting a video syntax from the video bitstream for the syntax type.17. The system as claimed in claim 16 wherein the decode module is forforming the video stream includes forming the video stream for aresolution greater than or equal to 7680 pixels by 4320 pixels.
 18. Thesystem as claimed in claim 16 wherein the decode module is forextracting the HRD syntax common to all occurrences of the temporallayer.
 19. The system as claimed in claim 16 wherein the decode moduleis for extracting an occurrence of the HRD syntax for each occurrence ofthe temporal layer.
 20. The system as claimed in claim 16 wherein theget syntax module is for setting the value of a coded picture buffercount based on the low delay flag.